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ARI  Research  Reports  and  Technical  Reports  are  intended  for  sponsors  of 
R&D  tasks  and  for  other  research  and  military  agencies.  Any  findings  ready 
for  implementation  at  the  time  of  publication  are  presented  in  the  last  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recom¬ 
mendations  for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


The  ARI  Surveys  of  Army  Recruits,  familiarly  known  as  New 
Recruit  Surveys,  have  become  an  important  source  of  information 
for  Army  policymakers  and  planners.  These  surveys  were  origi¬ 
nally  designed  in  1982  to  answer  questions  concerning  the  demo¬ 
graphics  and  enlistment  motivations  of  new  Army  recruits.  In 
addition  to  the  ability  to  track  changes  in  recruits  from  year 
to  year,  this  series  of  surveys  offers  the  Army  an  opportunity 
to  gather  new  information  in  each  year  of  the  survey. 

The  1982  and  1983  Surveys  were  directly  commissioned  by  the 
Deputy  Chief  of  Staff  for  Personnel.  The  1984  and  1985  Surveys 
were  sponsored  by  the  U.S.  Army  Recruiting  Command  and  were 
undertaken  with  the  continuing  interest  of  the  Office  of  the 
Deputy  Chief  of  Staff  for  Personnel.  While  the  sponsorship, 
specific  questions,  and  interests  may  change  year  to  year,  the 
charter  for  these  surveys  remains  the  same: 

e  Determine  who  is  enlisting  in  the  Army  and  why. 

•  Determine  how  to  target  recruiting  resources  to 

attract  high  quality  recruits. 

e  Determine  why  recent  recruits  joined  and  their 
propensity  to  remain  in  the  service. 

e  Determine  which  recruiting  and  advertising 
practices  are  proving  the  most  successful 
and  why. 

In  addition  to  addressing  these  specific  objectives,  this 
survey  effort  expands  the  availability  of  data  to  model  Army 
enlistment  and  reenlistment  processes. 
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THE  1984  AND  1985  ARI  SURVEY  OF  ARMY  RECRUITS: 

METHODOLOGY  AND  RECOMMENDATIONS  FOR  FUTURE  ADMINISTRATIONS 

EXECUTIVE  SUMMARY _ 


Requirement: 

New  Recruit  Surveys  provide  information  to  Army  personnel 
planners  regarding  the  characteristics,  knowledge  of  enlistment 
options,  and  enlistment  motivations  of  new  recruits.  It  is 
necessary  to  examine  the  methodologies  employed  in  recent  years 
(1984  and  1985)  to  develop  an  understanding  of  the  strengths  and 
weaknesses  of  each.  In  this  way,  future  efforts  can  take  advan¬ 
tage  of  the  experience  of  previous  NRS  administrations. 


Procedures : 

The  1984  Summer  and  Fall/Winter  New  Recruit  Survey  (NRS) 
were,  respectively,  the  fourth  and  fifth  in  a  series  of  recruit 
surveys  administered  by  ARI.  Data  were  collected  by  ARI  in  group 
settings  during  initial  recruit  processing  at  all  eight  Army 
Reception  Stations  during  June,  July,  August,  and  October  of 
1984,  as  well  as  February  1985.  Reception  Station  personnel  were 
responsible  for  scheduling  and  supervising  survey  administration, 
guided  by  instructions  prepared  by  ARI.  The  NRS  was  self- 
administered  and  recruits  were  directed  to  indicate  their 
responses  on  optical  scanning  forms. 

The  sixth  administration  of  the  NRS  was  conducted  in  the 
summer  of  1985.  The  1985  NRS  differed  from  previous  survey 
efforts  in  several  major  ways:  a  sampling  plan  was  developed 
which  distributed  administrations  across  the  summer  weeks  and 
Reception  Stations;  Westat  research  personnel  supervised  the 
survey  administrations  on-site;  and  sample  pools  of  respondents 
were  monitored  and  nonrespondents  rescheduled  for  the  survey. 

The  1984  NRS  administration  was  compared  with  the  1985  NRS 
in  terms  of  instrument  design,  data  collection,  and  data  prepara¬ 
tion  procedures.  Within  each  topic  area,  the  procedures  of  the 
1984  and  1985  surveys  are  briefly  reviewed  followed  by  a  discus¬ 
sion  of  the  lessons  learned  and  advantages/disadvantages  of  each. 
Based  upon  the  lessons  learned,  recommendations  are  made  for  the 
conduct  of  future  surveys. 
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Results 


A  number  of  recommendations  were  proposed  for  future  admin¬ 
istrations  of  the  NRS : 

•  The  survey  instrument  revision  process  should  be 
formalized; 

•  The  use  of  separate  answer  sheets  should  be  discon¬ 
tinued; 

•  NRS  administration  schedules  and  personnel  responsibil¬ 
ities  should  be  clearly  established  and  maintained; 
and, 

•  Actual  administration  of  the  NRS  should  be  supervised 
by  research  personnel,  not  Reception  Station  personnel. 


Utilization 


Design  of  the  Summer  1986  New  Recruit  Survey  (NRS)  benefited 
from  the  lessons  learned  in  the  1984  and  1985  NRS.  As  recom¬ 
mended  in  this  report,  plans  are  being  made  to  formalize  the 
survey  instrument  revision  process  in  the  1987  NRS.  The  lessons 
learned  and  recommendations  offered  in  this  report  provide  useful 
input  for  the  development  of  the  implementation  plan  for  the  U.S. 
Army  Recruiting  Command's  institutionalization  of  the  New  Recruit 
Survey. 
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THE  1984  AND  1985  ARI  SURVEY  OF  ARMY  RECRUITS: 
METHODOLOGY  AND  RECOMMENDATIONS  FOR  FUTURE  ADMINISTRATIONS 


INTRODUCTION 


The  purpose  of  this  report  is  to  overview  and  compare  the 
methodologies  employed  in  the  1984  (Summer  and  Fall/Winter)  and 
1985  (Summer)  administrations  of  the  U.S.  Army  Research  Institute 
Survey  of  New  Recruits,  present  lessons  learned  from  these 
administrations,  and  provide  recommendations  for  future  survey 
efforts.  This  first  chapter  introduces  other  reports  produced  to 
document  the  1984  and  1985  surveys,  discusses  the  organization  of 
this  final  report,  and  reviews  the  background  of  the  1984  and 
1985  NRS .  The  remaining  report  chapters  present:  (1)  lessons 
learned  from  the  two  administrations,  drawing  comparisons  between 
the  methodologies  employed,  and  (2)  recommendations  for  future 
surveys.  It  should  be  noted  that  the  1984  Summer  and  Fall/Winter 
administrations  are  treated  together,  i.e.,  the  1984  NRS,  when 
compared  to  the  1985  NRS.  This  approach  was  taken  because  the 
same  survey  instruments  were  used  in  the  1984  Summer  and  1984 
Fall/Winter,  and  the  survey  methodology  employed  in  both  admin¬ 
istrations  of  the  1984  NRS  were  nearly  identical. 


NRS  Reports 

This  task  final  report  is  one  in  a  series  of  20  reports 
produced  to  document  the  approach,  data,  and  results  obtained 
from  the  1984  and  1985  NRS.  (Other  reports  have  been  produced 
documenting  the  1982  and  1983  NRS.)  As  the  final  report,  this 
document  overviews  and  compares  the  survey  methodologies  used  in 
the  1984  and  1985  NRS,  including  instrument  design,  data  collec¬ 
tion,  data  preparation,  data  processing  techniques,  and  survey 
documentation  employed;  it  also  proposes  recommendations  for 
future  surveys.  The  recommendations  address  problems  inherent  in 
performing  ad  hoc  surveys  and  contrast  survey  outcomes  using 
different  implementation  strategies. 

In  order  to  fully  understand  the  lessons  learned  and  recom¬ 
mendations  presented  in  this  report,  readers  are  advised  to 
examine  the  three  User's  Manuals  developed,  respectively,  for  the 
two  1984  and  the  Summer  1985  administrations  of  the  NRS.  The 
User's  Manuals  document  the  survey  methodology  and  focus  on  pro¬ 
viding  background  information  for  survey  data  users.  The  Summer 
and  Winter  administrations  of  the  1984  NRS  used  the  same  adminis¬ 
tration  procedures  and  identical  versions  of  the  survey  instru¬ 
ment,  so  that  only  a  supplemental  User's  Manual  was  required  for 
the  Winter  administration.  As  the  name  implies,  the  Supplemen¬ 
tary  User's  Manual  provides  specific  information  about  the  Winter 
1984  NRS  that  complements  the  more  complete  1984  User's  Manual. 


The  User's  Manuals  provide  information  about  the  development  and 
content  of  the  NRS  instruments  and  survey  approaches  taken  in  the 
1984  and  1985  NRS  administrations.  Citations  for  these  three 
reports  are  as  follows: 


•  The  1984  ARI  Survey  of  Army  Recruits:  User's  Manual 
(Research  Note  86-46).  Alexandria,  VA:  US  Army 
Research  Institute  for  the  Behavioral  and  Social 
Sciences. 

•  The  1984  ARI  Survey  of  Army  Recruits:  Supplementary 
User's  Manual  for  Octobc~  84/Februarv  85  Administration 
(Research  Note  86-47).  Alexandria,  VA:  US  Army 
Research  Institute  for  the  Behavioral  and  Social 
Sciences . 

•  The  1985  ARI  Survey  of  Army  Recruits:  User's  Manual 
(Research  Note  86-50) .  Alexandria,  VA:  US  Army 
Research  Institute  for  the  Behavioral  and  Social 
Sciences . 

Other  reports  in  the  1984  and  1985  NRS  series  are  structured 
to  present  survey  results,  data  documentation,  and  survey  method¬ 
ology.  Codebooks  have  been  produced  to  document  the  physical 
locations  and  labels  for  all  variables  produced  on  the  SAS  and  OS 
survey  data  files  produced  for  the  1984  and  1985  NRS.  Each  of 
the  six  codebooks  pertains  to  a  particular  Army  service  component 
and  NRS  administration: 

•  The  1984  ARI  Survey  of  Army  Recruits:  Codebook  for 
Summer  84  Active  Army  Survey  Respondents  (Research 
Note  86-42).  Alexandria,  VA:  US  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

•  The  1984  ARI  Survey  of  Army  Recruits:  Codebook  for 
Summer  84  USAR  &  ARNG  Survey  Respondents  (Research 
Note  86-43).  Alexandria,  VA:  US  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

•  The  1984  ARI  Survey  of  Army  Recruits:  Codebook  for 
October  84/Februarv  85  Active  Army  Survey  Respondents 
(Research  Note  86-44).  Alexandria,  VA:  US  Army 
Research  Institute  for  the~'Behavioral  and  Social 
Sciences . 

•  The  1984  ARI  Survey  of  Army  Recruits:  Codebook  for 
October  84/Februarv  85  USAR  &  ARNG  Survey  Respondents 
(Research  Note  86-45).  Alexandria,  VA:  US  Army 
Research  Institute  for  the  Behavioral  and  Social 
Sciences . 


>  The  1985  ARI  Survey  of  Army  Recruits:  Codebook  for 
Summer  85  Active  Army  Survey  Respondents  (Research 
Note  86-48) .  Alexandria,  VA:  US  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

•  The  1985  ARI  Survey  of  Army  Recruits:  Codebook  for 
Summer  85  USAR  &  ARNG  Survey  Respondents  (Research 
Note  86-49) .  Alexandria,  VA:  US  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences: 

Preliminary  results  for  the  1984  and  1985  NRS  surveys  are 
reported  in  separate  documents  containing  crosstabulations  of  the 
NRS  items  by  selected  respondent  demographics  and  service-related 
characteristics.  The  tabulations  produced  for  the  Summer  1984 
and  1985  NRS  are  listed  below: 

•  The  1984  ARI  Survey  of  Army  Recruits:  Tabular 
Description  of  NPS  ( Active)  Army  Accessions.  Volumes  1 
and  2  (Research  Products  86-09  and  86-10) .  Alexandria 
VA:  US  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

•  The  1984  ARI  Survey  of  Army  Recruits:  Tabular 
Description  of  NPS  Army  National  Guard  Accessions 
(Research  Product  86-11) .  Alexandria,  VA:  US  Army 
Research  Institute  for  the  Behavioral  and  Social 
Sciences. 

•  The  1984  ARI  Survey  of  Army  Recruits:  Tabular 
Description  of  NPS  Army  Reserve  Accessions.  Volumes  1 
and  2  (Research  Products  86-12  and  86-13) .  Alexandria, 
VA:  US  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

•  The  1985  ARI  Survey  of  Army  Recruits:  Tabular 
Description  of  NPS  (Active)  Army  Accessions.  Volumes  1 
and  2  (Research  Products  86-14  and  86-15) .  Alexandria, 
VA:  US  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 


•  The  1985  ARI  Survey  of  Army  Recruits:  Tabular 
Description  of  Army  National  Guard  Accessions  (Research 
Product  86-16) .  Alexandria,  VA:  US  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

•  The  1985  ARI  Survey  of  Army  Recruits:  Tabular 

Description  of  NPS  Army  Reserve  Accessions.  Volumes  1 
and  2  (Research  Products  86-17  and  86-18) .  Alexandria, 
VA:  US  Army  Research  Institute  for  the  Behavioral  and 

Social  Sciences. 


No  comparable  volumes  were  produced  for  the  October  84/February 
85  survey  administrations.  The  same  crossing  variables  and 
recoded  survey  items  produced  for  the  1984  Summer  database  were 
also  constructed  for  the  Winter,  thus  allowing  data  analysts  to 
produce  tables  comparable  to  the  other  tabulation  volumes  using 
1984  Winter  data. 


Organization  of  the  Final  Report 

This  final  report  is  organized  into  three  chapters.  The 
first  introduces  the  report  within  the  context  of  the  twenty- 
volume  series  of  reports  produced  to  document  the  1984  Summer, 
1984  Winter,  and  1985  Summer  administrations  of  the  NRS  and 
discusses  the  structure  of  the  report  as  well.  The  first  chapter 
ends  with  an  overview  of  the  1984  and  1985  NRS  administrations. 

The  second  chapter  in  the  final  report  reviews  the  survey 
methodologies  employed  in  the  1984  and  1985  NRS,  drawing  compar¬ 
isons  of  the  instrument  design,  data  collection,  data  preparation 
and  processing  techniques,  and  survey  documentation.  The  advan¬ 
tages  and  disadvantages  of  each  approach  are  discussed,  and 
"lessons  learned"  from  these  administrations  are  presented. 

The  third  and  final  chapter  of  the  report  discusses  problems 
with  performing  ad  hoc  surveys.  The  chapter  also  focuses  on  the 
implications  of  administration  procedures  for  data  quality  and 
sample  representativeness.  Recommendations  for  future  surveys 
are  proposed. 


Project  Overview 

The  Army  Research  Institute  (ARI)  Surveys  of  Army  Recruits, 
commonly  known  as  the  ARI  New  Recruit  Surveys  (NRS) ,  are  con¬ 
ducted  on  a  regular  basis  to  provide  updated  information  on  the 
characteristics,  knowledge  of  enlistment  options,  and  enlistment 
motivations  of  recruits  at  the  point  of  their  initial  entry  into 
service.  Military  personnel  planners  require  such  information  on 
a  regular  basis  to  monitor  current  recruiting  strategies  and  to 
forecast  future  enlistment  trends. 

The  NRS  were  begun  in  the  spring  of  1982.  Expanded  versions 
of  the  1982  surveys  were  administered  in  the  Summer  of  1983  and 
Winter  of  1984.  Further  revisions  and  expansions  were  made  to 
the  1984  instruments  for  administration  in  the  Summer  of  1984  and 
Fall/Winter  of  FY  85.  The  NRS  was  further  expanded  for  the 
Summer  1985  administration.  In  addition  to  the  ability  to  track 
changes  in  recruits  from  year  to  year,  this  series  of  surveys 
offers  the  Army  an  opportunity  to  gather  new  information  in  each 
year  of  the  survey. 


The  1982  and  1983  surveys  were  directly  commissioned  by  the 
Deputy  Chief  of  Staff  for  Personnel.  The  1984  and  1985  NRS  were 
sponsored  by  the  U.S.  Army  Recruiting  Command  and  were  undertaken 
with  the  continuing  interest  of  the  Deputy  Chief  of  Staff  for 
Personnel.  While  the  sponsorship,  specific  questions,  and  inter¬ 
ests  may  change  from  year  to  year,  the  charter  for  these  surveys 
remains  the  same.  The  four  main  research  requirements  have  been 
to  determine:  (1)  the  demographic  composition  of  entering 
recruits;  (2)  how  to  target  recruiting  resources  to  attract  high 
quality  recruits;  (3)  new  recruits’  enlistment  motivations  and 
propensity  for  remaining  in  service;  and  (4)  which  recruiting  and 
advertising  practices  are  proving  the  most  successful  and  why. 

The  1984  Summer  and  Fall/Winter  NRS  are,  respectively,  the 
fourth  and  fifth  administrations  in  the  series  of  recruit  surveys 
conducted  by  ARI.  Identical  versions  of  the  NRS  instruments  were 
administered  to  recruits  in  the  two  administrations.  The  1984 
survey  was  an  expanded  version  of  previous  NRS.  No  formal 
sampling  plan  was  implemented  for  the  1984  NRS,  however,  it  was 
expected  that  all  recruits  being  processed  for  initial  entry 
training  during  the  administration  period  would  be  surveyed  by 
the  Reception  Stations.  Survey  data  were  collected  in  group 
settings  during  initial  recruit  processing  at  all  eight  Army 
Reception  Stations  during  June-August,  1984,  and  again  in  October 
1984  and  February  1985.  Reception  station  personnel  were  respon¬ 
sible  for  scheduling  and  supervising  survey  administration, 
guided  by  instructions  prepared  by  ARI.  The  NRS  was  self- 
administered  and  recruits  were  directed  to  indicate  their 
responses  on  optical  scanning  forms. 

The  sixth  administration  of  the  NRS  was  conducted  in  the 
Summer  of  1985,  and  reflected  numerous  changes  in  both  the  design 
of  the  instrument  and  survey  administration  procedures.  As  in 
previous  years,  this  version  of  the  NRS  incorporated  the  revision 
of  certain  questions  and  the  addition  of  others  to  reflect 
changes  in  both  personnel  planning  concerns  and  the  recruiting 
environment.  New  questions  in  this  instrument  tended  to  focus  on 
experiences  and  influences  occurring  before  entry  into  the  Army, 
to  allow  assessment  of  their  effect  on  enlistment  decision 
making.  One  major  change  in  the  1985  NRS  instrument  was  that 
respondents  answered  survey  items  directly  on  the  instrument. 

All  previous  NRS  surveys  used  separate  optical  scanning  answer 
sheets. 

The  Summer  1985  NRS  administration  differed  from  previous 
survey  efforts  in  several  important  ways.  First,  the  1985  NRS 
used  a  sampling  plan  to  allocate  administrations  evenly  through¬ 
out  the  survey  period.  Two  Reception  Stations  were  surveyed 
during  each  of  the  twelve  administration  weeks,  during  the  months 
of  June-September,  1985.  Second,  Reception  Station  personnel 
were  not  responsible  for  survey  administration.  On-site  supervi¬ 
sion  and  administration  of  the  NRS  was  the  responsibility  of 


research  personnel  from  Westat,  Inc.  Third,  lists  were  main¬ 
tained  of  recruits  beginning  initial  processing  at  each  of  the 
eight  Reception  Stations.  In  order  to  achieve  high  response 
rates,  efforts  were  made  to  keep  track  of  and  reschedule  new 
recruits  who  failed  to  complete  a  survey  form. 

During  both  the  1984  and  1985  NRS ,  four  forms  of  the  NRS 
were  administered  to  new  recruits.  Forms  A,  B,  and  C  were 
administered  to  Regular  Army  recruits,  and  Form  D  was  given  to 
National  Guard  and  Army  Reserve  recruits.  Regular  Army  recruits 
had  an  equal  chance  of  receiving  any  one  of  the  three  forms. 

A  total  of  15,309  Army  recruits  completed  the  1984  NRS. 
During  the  Summer  administration,  10,495  of  these  recruits  were 
surveyed,  while  4,814  recruits  completed  surveys  during  the 
Fall/Winter  1984  NRS.  The  1985  NRS  surveyed  a  total  of  12,536 
Army  recruits.  The  surveys  were  distributed  across  components 
and  administrations  as  follows: 


NRS 


Regular  Army  Army  Reserve/National  Guard 


Summer  1984 
Winter  1984 
Summer  1985 


6,184 

3,524 

7,220 


4,311 

1,290 

5,316 


The  1984  NRS  answer  sheets  were  optically  scanned.  The  1985 
survey  responses  were  reviewed  by  a  coding  staff  and  prepared  for 
data  entry.  Care  was  taken  in  both  instances  to  record  responses 
exactly  as  marked.  Survey  data  were  then  edited  for  value  ranges 
and  logical  consistency.  User's  manuals,  codebooks,  and  tabula¬ 
tion  volumes  were  produced. 

The  sampling  plans  employed  impose  some  restrictions  on  the 
general inability  of  survey  results  to  the  populations  of  Army 
accessions  during  FY  84  and  FY  85.  However,  overall,  these 
survey  efforts  were  successful  in  obtaining  updated  information 
regarding  new  recruits. 


The  1984  and  1985  ARI  Surveys  of  Army  Recruits  provide 
information  to  Army  personnel  planners  about  the  quantity  and 
quality  of  recruits  entering  the  Army.  In  particular,  informa¬ 
tion  regarding  the  characteristics,  knowledge  of  enlistment 
options,  and  enlistment  motivations  "df  new  recruits  enables  Army 
personnel  planners  to  be  cognizant  of  present  enlistment  trends. 
Awareness  of  these  trends  is  vital,  in  turn,  for  preparing  for 
future  manpower  requirements. 
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LESSONS  LEARNED 


This  chapter  reviews  and  compares  the  NRS  1984  and  1985 
administrations.  Details  of  administration  and  specific  lessons 
learned  in  each  administration  are  documented  more  fully  in  the 
separate  User's  Manuals  for  1984  (Research  Notes  86-46  and  86-47) 
and  1985  (Research  Note  86-50) .  This  chapter  focuses  on  what  can 
be  learned  from  differences  in  outcomes  resulting  from  differ¬ 
ences  in  survey  procedures.  There  are  three  main  areas  of  survey 
administration  covered  in  this  discussion:  (1)  instrument 
design,  (2)  survey  administration  and  data  collection,  and 
(3)  data  preparation  and  processing.  Each  of  these  areas  is 
covered  in  a  separate  subsection  in  this  chapter. 


Instrument  Design 


In  many  respects,  the  1984  and  1985  NRS  instruments  are  very 
similar.  Both  can  be  considered  refinements  and  expansions  of 
earlier  NRS  instruments.  The  content  of  the  instruments  designed 
in  each  year  covered  similar  topics.  Both  the  1984  and  1985  NRS 
used  multiple  forms:  three  versions  of  the  RA  instrument,  and  a 
combined  survey  form  for  the  USAR  and  ARNG.  There  was  a  similar 
focus  in  both  years  on  obtaining  responses  from  non-prior  service 
recruits  and  not  exceeding  a  time  limitation  of  one  hour  for 
survey  completion. 


One  major  difference  in  the  survey  instruments  employed  in 
each  of  the  two  years  was  that  the  1984  NRS  was  designed  for  use 
with  an  OPSCAN  answer  sheet  and  the  1985  was  formatted  for  entry 
of  responses  directly  on  the  instrument.  This  difference  in 
questionnaire  format  had  implications  for  the  types  and  struc¬ 
tures  of  questions  that  could  be  included  in  each  instrument.  In 
addition,  differences  in  the  1984  and  1985  survey  formats  led  to 
considerable  differences  in  data  handling  procedures,  which  will 
be  discussed  later  in  this  chapter. 


The  use  of  OPSCAN  answer  sheets  placed  a  number  of  restric¬ 
tions  on  the  design  of  the  1984  instruments.  The  number  of  ques¬ 
tions  that  could  be  included  on  each  questionnaire  was  limited  to 
the  number  of  lines  available  on  the  answer  sheets.  Skip  pat¬ 
terns  were  not  able  to  be  used  in  the  instrument  since  respon¬ 
dents  can  easily  lose  their  place  on  the  questionnaire  while 
writing  down  answers  on  a  separate  answer  sheet.  This  meant  that 
questions  which  were  inappropriate  for  certain  classes  of  respon¬ 
dents  had  to  include  response  categories  that  indicated  the  ques¬ 
tion's  inapplicability  for  these  individuals.  This  probably 
contributed  to  the  frustration  of  respondents  who  were  forced  to 
read  all  questions  even  when  they  were  inappropriate.  Another 
problem  encountered  was  that  questions  requiring  multiple 
responses,  e.g.,  "mark  all  (response  categories)  that  apply" 
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formatted  questions,  needed  special  instructions  for  respondents 
to  input  all  answers  in  the  same  answer  slot.  This  was  counter 
to  the  typical  procedure  of  recording  a  single  response  in  a 
given  line  on  the  answer  sheet.  It  is  unknown  how  many  respon¬ 
dents  became  confused  and  used  several  answer  lines  to  fill  in 
responses  to  a  single  multiple-part  question. 

The  use  of  survey  instruments  that  allowed  the  recording  of 
responses  directly  on  the  form  in  the  1985  NRS ,  provided  more 
flexibility  than  the  use  of  both  a  questionnaire  and  separate 
answer  sheet.  The  switch  to  this  format  enabled  the  inclusion  of 
open-ended  questions  requesting  both  short  and  long  written 
responses.  This  was  a  considerable  advantage  in  terms  of  obtain¬ 
ing  dollar  figures,  dates,  the  names  of  schools  attended,  courses 
of  study  and  so  on.  In  addition,  this  format  made  it  possible  to 
explore  new  topic  areas  with  respondents  without  the  necessity  of 
conducting  qualitative  interviews.  Analysts  can  study  answers  to 
the  open-ended  questions  and  obtain  a  better  understanding  of  the 
perspectives  of  new  recruits  on  a  variety  of  previously  unex¬ 
plored  subjects  and  obtain  a  more  in-depth  view  of  the  rationale 
behind  recruits'  answers  to  related  pre-coded  questions.  Because 
answers  were  recorded  on  the  same  page  as  the  printed  questions, 
multiple  response  questions  were  easily  completed.  Skip  patterns 
were  incorporated  in  the  1985  NRS  instruments  to  eliminate  the 
need  for  respondents  to  read  questions  that  were  obviously  inap¬ 
propriate,  given  their  answers  to  previous  questions.  This 
avoided  the  problem  of  having  respondents  read  through  inapplica¬ 
ble  questions  and  their  answer  categories  in  an  attempt  to  find  a 
response  indicating  that  the  question  was  inappropriate.  The  use 
of  skip  patterns  should  have  aided  in  lowering  respondents' 
frustration  levels  and  thus  response  error  rates.  The  number  of 
questions  that  could  be  asked  in  the  1985  NRS  was  not  limited  by 
the  use  of  a  separate  answer  sheet  as  it  had  been  in  the  1984 
NRS.  The  only  limitation  on  the  number  of  questions  that  could 
be  included  in  the  1985  NRS  was  the  requirement  that  the  survey 
not  take  longer  than  60  minutes  to  complete.  Thus,  the  inclusion 
of  a  large  number  of  questions  that  required  minimal  time  to 
answer  posed  no  particular  problem  for  the  1985  NRS. 

Although  many  useful  formatting  techniques  were  incorporated 
to  facilitate  survey  completion  by  respondents,  there  were  still 
some  problems  associated  with  the  instrumentation  of  the  1985 
NRS.  There  was  minimal  time  allotted  for  questionnaire  design 
and  review  and  no  arrangements  for  a  pretest.  This  led  to  the 
need  for  instrument  revisions  during  the  first  week  of  survey 
fielding.  Instruments  were  pulled  out  of  printing  production  for 
last  minute  question  additions  and  response  category  revisions. 
This  resulted  in  difficulties  during  the  data  preparation  stage 
because  special  missing  value  codes  had  to  be  devised  to  identify 
cases  in  which  the  new  questions  did  not  appear  and  in  which  new 
response  categories  were  not  present.  Selected  coders  were 
trained  to  handle  the  early  versions  of  the  survey  instruments. 


This  special  handling  was  necessary  in  order  to  avoid  developing 
separate  formats  for  data  entry  (see  the  1985  User's  Manual, 
Research  Note  86-50) . 


A  pretest  would  have  helped  to  identify  additional  problems 
encountered  by  respondents  in  the  1985  NRS.  For  example,  it  was 
discovered  that  respondents  generally  had  difficulty  in  recording 
the  dollar  amounts  that  accrued  with  various  educational  incen¬ 
tives,  in  terms  of  both  their  own  contribution  amounts  as  well  as 
contribution  amounts  from  the  government.  Also,  respondents  who 
were  surveyed  early  in  their  in-processing  were  less  likely  to 
accurately  recall  information  regarding  enlistment  incentives 
than  respondents  who  were  nearing  the  end  of  their  processing  at 
the  Reception  Stations.  This  is  likely  due  to  the  fact  that  the 
Reception  Stations  provide  both  general  discussions  regarding 
Army  enlistment  incentives  as  well  as  individual-specific  dis¬ 
cussions  about  incentives  eligibility  near  the  end  of  initial 
processing.  Thus,  it  is  important  to  have  respondents  record  the 
day  of  Reception  Station  processing  they  are  in  at  the  time  of 
survey  administration.  This  will  enable  analysts  to  examine 
differences  in  the  knowledge  levels  and  attitudes  of  recruits  in 
various  stages  of  in-processing. 


Survey  Administration  and  Data  Collection 

1984  NRS.  It  is  somewhat  difficult  to  contrast  the  actual 
survey  administration  procedures  used  in  the  1984  with  those 
employed  for  the  1985  NRS  because  the  former  was  conducted  by 
personnel  at  each  of  the  eight  Reception  Stations  guided  by 
instructions  prepared  by  ARI  (see  Appendix  E  in  the  1984  User's 
Manual,  Research  Note  86-46).  The  instructions  prepared  by  ARI 
covered  activities  pertaining  to  three  main  phases  of  the  NRS: 

(1)  pre-administration  preparation,  (2)  administration  proce¬ 
dures,  and  (3)  preparation  for  subsequent  groups.  Pre¬ 
administration  preparation  instructions  requested  that  the  survey 
administrators  familiarize  themselves  with  the  survey  booklets 
and  contact  ARI  personnel  to  resolve  any  difficulties.  In 
addition,  administrators  were  asked  to  make  available  adequate 
seats  and  table  space,  as  well  as  pencils  with  #2  lead  for  each 
survey  administration.  Administrators  were  asked  to  display  the 
survey  date  and  site  number  for  respondents,  to  ensure  that 
respondents  brought  processing  forms-containing  their  social 
security  numbers  to  the  survey,  and  to  distribute  answer  sheets, 
survey  booklets,  and  pencils  prior  to  seating  recruits  for  the 
NRS. 


ARI  instructions  regarding  survey  administration  procedures 
requested  that  survey  administrators  check  to  be  sure  that 
recruits  filled  out  the  survey  forms  appropriate  to  their 
component,  read  the  Privacy  Act  of  1973  and  ARI  assurances  of 
confidentiality,  and  read  instructions  pertinent  to  the  use  of 


optical  scan  answer  sheets.  Survey  administrators  were  then 
asked  to  read  aloud  to  recruits  instructions  for  completing  the 
background  section  of  the  survey  (e.g.,  fill  in  social  security 
numbers,  sex,  survey  date,  and  so  on)  and  to  read  final  instruc¬ 
tions  regarding  skipping  questions,  time  allotted  for  completing 
the  survey,  and  how  to  turn  in  materials  at  the  end  of  the 
administration.  Survey  proctors  were  requested  to  remain  in  the 
room  with  respondents  throughout  the  survey,  attend  to  individual 
questions  by  going  to  the  recruit's  seat,  and  to  thank  each 
soldier  for  completing  the  survey  as  they  turned  in  their 
materials . 

Post-administration  procedures  included  review  of  each 
instrument  for  marking,  discarding  those  which  had  written  com¬ 
ments,  separating  the  USAR/ARNG  forms  from  the  three  RA  surveys, 
and  interleaving  Forms  A,  B,  and  C  in  a  systematic  way  (i.e., 

A, B, C , A, B, C) . 

Administration  procedures  varied  somewhat  at  each  of  the 
eight  Reception  Stations  as  Personnel  Affairs  administered  the 
NRS-84  at  some  installations  and  at  others  the  Testing  Branch 
handled  the  survey  procedures.  Examination  of  Table  3  in  the 
1984  User's  Manual  (Research  Note  86-46)  and  Table  2  in  the 
October  84/February  85  Supplementary  User's  Manual  (Research  Note 
86-47)  reveal  that  administration  schedules  were  not  followed 
uniformly  across  installations.  ARI  had  requested  that  the  NRS- 
84  survey  administrations  be  carried  out  during  the  weeks  of 
25  June,  16  July,  and  6  August,  1984  for  the  summer  administra¬ 
tions,  and  during  the  weeks  of  29  October,  1984  and  18  February, 
1985.  In  fact,  some  Reception  Stations  administered  the  NRS-84 
during  weeks  prior  to  and  others  later  than  those  assigned,  some 
installations  administered  the  surveys  across  two  different 
weeks,  and  others  failed  entirely  to  administer  the  survey  during 
the  assigned  months.  The  differences  in  the  administration 
schedules  were  fairly  easy  to  detect,  other  differences  in  the 
administration  procedures  may  have  been  more  subtle  and  less  easy 
to  identify.  For  example,  it  is  not  known  how  closely  Reception 
Station  personnel  followed  the  set  of  instructions  prepared  for 
them  by  ARI.  It  is  likely  that  other  conditions  of  survey 
administration  differed  across  installations.  Perhaps  some 
administrators  failed  to  familiarize  themselves  with  the  survey 
instruments  and  were  thus  unable  to  answer  questions  as  they 
arose  in  the  survey  sessions;  at  other  installations,  survey 
administrators  probably  gave  their  own  interpretation  of  what 
specific  survey  items  were  asking.  Some  Reception  Stations  prob¬ 
ably  had  sergeants  or  officers  in  attendance  at  the  survey 
administration  to  maintain  order  and  no  uniformed  personnel 
present  at  others.  This  may  not  necessarily  affect  survey 
results  unless  respondents  perceived  the  uniformed  personnel  to 
be  "watching  over"  the  answers  they  were  providing. 
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A  major  difficulty  in  having  Reception  Station  personnel 
administer  the  NRS  is  that  there  is  little  or  no  documentation 
produced  that  will  allow  researchers  to  know  what  problems  came 
up,  if  any,  and  how  they  were  handled.  The  main  reference  source 
that  was  maintained  by  installation  personnel  was  the  log-in 
sheets.  These  sheets  indicated  how  many  recruits  were  each  day's 
processing  load,  the  number  surveyed,  the  number  declining  to 
take  the  survey,  and  the  number  not  surveyed  for  other  reasons. 
Had  this  information  been  faithfully  and  accurately  provided  by 
all  survey  administrators,  this  would  have  provided  the  informa¬ 
tion  that  was  needed  to  calculate  response  rates  for  each  Recep¬ 
tion  Station.  Unfortunately,  these  forms  were  not  completed 
uniformly  nor  completely  for  the  various  administrations  of  the 
1984  NRS. 

1985  NRS.  In  the  spring  of  1985,  ARI  sent  letters  of 
introduction  regarding  the  upcoming  NRS  and  requested  a  point  of 
contact  (POC)  at  each  Reception  Station.  This  initial  letter 
indicated  that  Westat  research  staff  would  be  conducting  the 
administration  of  the  summer  1985  NRS.  Westat  staff  contacted 
Reception  Station  POC's  to  set  up  survey  arrangements  and  to 
inform  them  of  the  administration  schedule.  At  the  first  visit, 
an  ARI  staff  member  accompanied  the  Westat  survey  administrator 
to  the  installation  and  provided  the  POC  and  other  interested 
parties  with  an  in-briefing  regarding  the  NRS.  This  set  up  an 
opportunity  for  the  Reception  Station  staff  to  ask  questions, 
clarify  responsibilities,  and  to  develop  a  better  understanding 
of  the  purpose  and  importance  of  the  New  Recruit  Surveys. 

Westat  staff  set  up  survey  materials,  reviewed  the  lists  of 
incoming  recruits,  checked  the  daily  schedule  of  survey  adminis¬ 
trations,  and  conducted  the  daily  survey  administrations. 
Typically,  uniformed  NCO's  were  available  to  proctor  the  survey 
and  to  maintain  order  as  recruits  completed  their  survey  forms. 
The  introductory  comments  and  survey  instructions  were  structured 
in  advance  to  ensure  completeness  and  consistency  of  information 
in  the  administrations  across  the  installations.  A  brief 
statement  concerning  the  Privacy  Act  of  1974  was  read  to  each 
group  of  recruits  prior  to  their  taking  the  survey.  Westat 
personnel  collected  the  survey  materials  after  each  administra¬ 
tion  and  checked  off  those  completing  the  survey  and  listed 
nonrespondents;  installation  POC's  were  notified  of  "no-shows" 
and  were  asked  to  reschedule  an  administration  time  for  these 
individuals.  Westat  staff  took  responsibility  for  safely 
delivering  the  complete  surveys  and  sample  lists  back  for  data 
preparation. 

There  were  a  number  of  procedures  employed  in  the  1985  NRS 
administration  that  could  be  altered  in  future  surveys  to  improve 
survey  effectiveness.  One  such  procedure  involves  the  use  of 
government  travel  orders.  Monies  for  travel  to  and  per  diem  at 
the  field  sites  came  directly  from  ARI  and  was  not  a  part  of  the 
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contract  funds.  This  required  an  enormous  amount  of  coordination 
between  staff  at  ARI  and  Westat  regarding  the  scheduling  of 
travel  arrangements.  Use  of  travel  orders  also  limited  the  flex¬ 
ibility  that  was  needed  to  carry  out  an  extensive  field  effort, 
especially  in  terms  of  juggling  personnel  schedules.  In  order  to 
avoid  the  problems  involved  in  having  one  office  obtain  the 
orders  needed  to  arrange  travel  for  another  group's  personnel, 
and  to  ensure  flexibility  for  attaining  field  work  goals,  it 
would  be  best  to  include  travel  monies  in  future  NRS  budgets. 

A  second  procedure  that  could  be  easily  altered  involves  the 
identification  of  installation  POC's.  In  some  instances,  partic¬ 
ularly  when  a  civilian  had  been  identified  as  the  sole  point  of 
contact,  it  was  apparent  that  the  NRS  survey  administrator  needed 
access  to  an  additional  POC  directly  involved  in  the  scheduling 
of  in-processing  operations.  It  would  seem  useful  in  future  NRS 
administrations  to  locate  a  POC  within  in-processing  operations 
as  well  as  an  individual  involved  with  the  Reception  Station 
Personnel  Affairs  or  Testing  Branch. 


Data  Preparation 

There  are  two  main  issues  linked  to  discussions  of  the  data 
preparation  procedures  carried  out  in  the  1984  and  1985  NRS. 

These  issues  are  related  to  the  costs  involved  in  getting  the 
data  to  machine-readable  format  and  the  exercise  of  control  over 
this  process.  The  1984  NRS  employed  optical  scan  (OPSCAN)  answer 
sheets  and  the  1985  NRS  had  respondents  provide  their  answers 
directly  on  the  questionnaire  itself.  These  two  approaches 
necessitated  some  differences  in  the  procedures  used  to  translate 
responses  into  machine-readable  format.  The  1984  answer  sheets 
were  forwarded  to  an  optical  scanning  company  for  processing 
while  the  1985  responses  were  processed  by  a  coding  operation, 
were  keyed  onto  tape,  and  then  edited  using  a  combination  of 
manual  and  automated  techniques. 

There  are  both  positive  and  negative  points  that  can  be  made 
regarding  the  use  of  OPSCAN  answer  sheets.  Positive  comments 
relate  to  cost.  Typically,  it  is  less  costly  to  translate  survey 
responses  that  have  been  entered  on  OPSCAN  answer  sheets  to  com¬ 
puter  tape  than  it  is  to  use  coding  and  editing  operations  with 
key  entry.  However,  there  are  tradeoffs  associated  with  this 
cost  savings.  OPSCAN  processing  usually  means  "What  the  machine 
sees  is  what  you  get."  The  assumption  is  made  that  the  form  was 
completed  properly  and  the  information  obtained  is  unambiguous. 
These  assumptions  may  or  may  not  be  valid. 

On  the  other  hand,  a  coding  and  editing  operation  is  much 
more  likely  to  be  able  to  make  interpretations  of  what  respon¬ 
dents'  intentions  were  in  cases  of  erasures,  hand-written  com¬ 
ments  entered  in  the  margins,  light  (pale)  writing,  and  answers 


written  with  other  than  a  Number  2  pencil  than  an  optical 
scanning  operation.  In  addition,  coders  can  distinguish  between 
different  types  of  missing  or  invalid  data  (e.g.,  blanks,  multi¬ 
ple  responses,  out-of-range  responses,  etc.)  and  are  trained  to 
look  for  unusual  and  inconsistent  response  patterns  in  the  data 
as  they  code.  The  tradeoff  for  coding  and  editing  operations  is 
that  they  typically  cost  considerably  more  than  OPSCAN  process¬ 
ing. 

The  choice  between  OPSCAN  and  coding/editing  operations  also 
has  an  impact  on  data  processing.  Coding/editing  operations  can 
reduce  the  number  and  complexity  of  subsequent  machine  editing 
checks  required  after  the  data  have  been  keyed  by  making  correc¬ 
tions  before  the  questionnaires  are  sent  to  data  entry.  This  is 
not  normally  the  case  with  optical  scanning  procedures.  The  pro¬ 
cessing  of  the  data  is  entirely  in  the  hands  of  the  optical  scan¬ 
ning  company  and  the  job  is  fit  into  their  work  schedule. 

One  additional  area  of  concern  in  using  the  OPSCAN  method 
for  the  NRS  concerns  the  match  of  methodology  to  respondent  moti¬ 
vation.  Typical  optical  scanning  applications  record  the  answers 
of  individuals  taking  standardized  achievement  tests  such  as 
those  required  for  college  and  military  service  applications.  In 
such  instances,  respondents  have  high  motivation  to  fill  out  the 
answer  sheet  accurately  since  the  test  results  will  be  used  to 
help  determine  whether  or  not  they  will  be  accepted  in  college  or 
the  Army.  However,  NRS  respondents  are  being  asked  to  complete  a 
non-mandatory  survey  in  a  new  and  confusing  atmosphere.  At  the 
Reception  Stations  the  new  recruits'  days  are  hectic  and  heavily 
scheduled.  The  NRS  administrations  often  had  to  be  scheduled 
either  at  the  beginning  or  end  of  recruits'  days  at  the  process¬ 
ing  stations.  This  set  of  circumstances  may  have  contributed  to 
low  motivation  to  carefully  complete  the  surveys,  particularly 
one  which  required  the  additional  effort  of  recording  one's 
answers  on  a  separate  sheet  from  where  the  questions  were 
printed.  There  are  some  indications  (compare  the  response  error 
rates  in  the  two  1984  and  the  1985  User's  Manuals,  and  see 
Table  1  in  the  next  chapter)  that  response  error  rates  were 
reduced  with  the  introduction  of  the  1985  NRS  instruments. 
Presumably  these  forms  were  easier  for  respondents  to  complete. 

The  next  chapter  of  this  report  will  examine  the  lessons 
learned  in  the  1984  and  1985  NRS  and^will  make  recommendations 
that  may  prove  useful  in  guiding  future  survey  efforts  of  this 
sort.  Some  additional  comparisons  between  the  survey  formats 


■^It  is  also  true  that  optical  scanning  companies  can  also  perform 
many  of  these  functions  (including  resetting  the  readers  to  pick 
up  light  marks)  but  such  increased  processing  drives  up  the  cost 
and,  in  the  case  of  resetting  the  sensitivity  of  the  reader, 
will  likely  introduce  error. 


used  in  the  1984  and  1985  NRS  such  as  instrument  flexibility, 
data  quality  and  documentation,  turnaround  time,  and  cost  are 
drawn  in  this  chapter. 


RECOMMENDATIONS  FOR  FUTURE  NRS  ADMINISTRATIONS 


This  chapter  presents  recommendations  for  future  NRS 
administrations.  The  recommendations  are  a  direct  outgrowth  of 
lessons  learned  from  the  1984  and  1985  NRS  administrations. 

Three  major  topic  areas  are  covered  in  this  chapter.  They  are: 

(1)  the  survey  instrument,  (2)  survey  administration,  and 
(3)  database  construction. 

The  recommendations  offered  are  presented  as  ways  to  improve 
future  NRS  administrations  and  insure  the  collection  of  high 
quality  survey  data.  Some  recommendations  identify  areas  where 
ad  hoc  procedures  should  be  replaced  with  a  more  formal  implemen¬ 
tation  while  others  suggest  methods  for  improving  data  quality. 


The  NRS  Survey  Instrument 

The  continually  changing  nature  of  the  recruiting  environ¬ 
ment  makes  it  necessary  to  revise  NRS  questions  from  time-to- 
time.  This  updating  of  the  NRS  constitutes  one  of  its  strengths. 
Such  revisions  allow  the  NRS  to  address  contemporary  conditions 
and  issues.  The  revision  process,  however,  has  not  always 
proceeded  smoothly.  We  offer  two  recommendations  for  future  NRS 
revisions: 

•  The  revision  process  should  be  formalized.  Revisions 
should  be  made  according  to  a  schedule  which  is  firmly 
set  and  monitored. 

•  The  use  of  separate  answer  sheets  should  be  discon¬ 
tinued.  Recruits  should  record  their  responses  on  the 
survey  instrument. 

In  addition  to  these  recommendations,  a  discussion  of  question¬ 
naire  format  is  presented.  Specifically,  the  tradeoffs  involved 
between  adoption  of  an  OPSCAN  or  more  traditional  questionnaire 
format  are  indicated. 

Formalize  Questionnaire  Revision  and  Design.  During  the 
1985  NRS,  the  questionnaire  redesign  process  was  not  completed 
until  the  first  week  in  June.  This  "necessitated  rescheduling  the 
first  week  of  administration.  Several  minor  wording  problems 
were  discovered  with  the  survey  instruments  which  were  fielded  in 
the  second  week  of  June  1985.  This  resulted  in  a  second  round  of 
last  minute  revisions  before  the  final  1985  NRS  instruments  were 
printed.  Thus,  in  addition  to  requiring  changes  in  administra¬ 
tion  schedules,  the  late  delivery  of  the  survey  instrument 
prevented  instrument  pretesting  which,  in  turn,  caused  numerous 
problems  during  NRS  database  construction  and  necessitated  survey 
item  reconciliation. 


Future  NRS  instrument  design  and  revision,  including 
instrument  pretesting  and  revision,  should  be  accomplished  in 
accordance  with  a  strictly  monitored  timetable.  The  precise 
structure  of  the  revision  and  design  process  is  less  important 
than  the  establishment  of  decision  making  authority  regarding 
revisions  and  a  timetable  for  decisions.  Deadlines  for  sub¬ 
mission  of  suggested  revisions,  response  to  suggestions,  and 
questionnaire  construction  must  be  set  and  followed.  Regardless 
of  the  actual  process  used,  the  final  version  of  the  NRS  ques¬ 
tionnaire  should  be  available  no  later  than  one  month  prior  to 
the  first  scheduled  administration.  This  allows  a  reasonable 
period  for  the  printing,  and  distribution  of  questionnaires. 

One  suggestion  that  follows  from  the  very  extensive  data 
cleaning  procedures  performed  in  the  1985  NRS  would  be  to  under¬ 
take  an  item  analysis  before  developing  any  future  surveys.  The 
1985  NRS  was  very  thorough  in  documenting  various  types  of 
response  errors  such  as  missing  data,  out-of-range  responses, 
inappropriate  multiple  responses,  and  inconsistent  responses  to 
related  variables,  as  well  as  labeling  other  special  missing 
value  codes,  i.e.,  valid  skip  patterns,  non-matches  with  Army 
personnel  records.  Future  designers  of  NRS  instruments  would  be 
well-advised  to  examine,  in  a  complete  and  systematic  manner,  the 
validity  of  survey  items  before  incorporating  previously  used 
questions  in  additional  surveys.  For  example,  it  is  clear  that 
some  respondents  encountered  difficulties  in  attempting  to  record 
the  dollar  amounts  of  the  government's  and  their  contributions 
for  various  educational  benefits,  as  well  as  enlistment  bonus 
amounts,  and  so  on.  Also,  some  survey  items  may  have  posed  dif¬ 
ficulties  for  other  respondents  as  demonstrated  in  the  numbers  of 
failed  consistency  checks,  the  large  number  of  out-of-range 
responses,  multiple  response  errors,  and/or  missing  data.  Analy¬ 
sis  of  the  configuration  of  response  errors  for  each  survey  item 
in  the  1985  NRS  would  aid  in  the  design  of  future  surveys 
incorporating  questions  on  similar  topics. 

Discontinue  Use  of  Separate  Answer  Sheets.  During  the  1984 
NRS,  recruits  recorded  their  answers  to  survey  items  on  separate 
answer  sheets.  It  is  recommended  that  this  p  actice  be  discon¬ 
tinued.  Table  1  presents  data  quality  indicators  for  the  two 
1984  administrations  and  the  single  1985  administration. 
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Table  1 


Total  Response  Error  Percentages 
by  NRS  Administration* 

Number  of 
Errors 

j  Summer  1984 

1  % 

Fall/Winter  1984 
% 

Summer  1985 
% 

None 

|  3.3 

3.6 

10.5 

10  or  More 

|  36.3 

40.0 

29.0 

*  Source:  Table  12,  1984  User's  Manual,  Research  Note  86-46; 
Table  11,  Supplementary  User's  Manual  for  October  84/ 

February  85  Administration,  Research  Note  86-47;  and  Table  14, 
1985  User's  Manual,  Research  Note  86-50. 


Table  1  illustrates  dramatic  differences  in  response  error 
percentages  associated  with  response  format.  In  the  1985  NRS 
recruits  recorded  their  responses  directly  on  the  survey 
instrument  rather  than  on  a  separate  answer  sheet  as  in  the  1984 
NRS.  The  percentage  of  surveys  having  no  response  errors  nearly 
tripled  in  1985  while  the  percentage  with  10  or  more  errors  is 
significantly  lower  than  that  in  1984. 

OPSCAN  versus  Traditional  Survey  Format.  The  recommenda¬ 
tions  made  regarding  the  NRS  survey  instrument  raise  a  number  of 
survey  design  and  processing  issues  that  impact  upon  the  NRS.  We 
consider  here  only  those  issues  related  to  the  operations 
required  for  creating  a  raw  (i.e.,  unedited)  survey  database. 
Among  the  issues  that  shape  survey  design  and  processing  are  the 
following: 

•  Flexibility  -  are  all  survey  items  to  be  precoded  or  is 
it  necessary  to  include  open-ended  questions? 

•  Data  Quality  and  Documentation  -  how  fully  documented 
and  free  from  error  must  survey  data  be? 

•  Turnaround  Time  -  how  soon  following  survey  administra¬ 
tion  must  the  data  become  available  for  analysis? 


Cost  -  what  level  of  effort  is  the  sponsoring  agency 
willing  to  support? 


These  issues  are  obviously  interdependent.  Nonetheless,  in 
planning  an  NRS  administration,  each  must  be  considered.  The 
tradeoffs  within  each  issue  will  be  illustrated  using  the  1984 
and  1985  NRS  administrations  as  examples  of  fairly  standard 
OPSCAN  and  traditional  survey  implementations.  The  options 
available  for  survey  format  and  processing  are  not  restricted  to 
those  used  in  1984  and  1985,  however.  Both  OPSCAN  and  tradi¬ 
tional  survey  procedures  may  be  modified  to  accommodate  a  variety 
of  features.  Accordingly,  this  subsection  will  close  with  a 
consideration  of  various  hybrid  designs  that  may  prove  helpful  in 
the  future. 

How  flexible  must  the  instrument  be?  The  1984  NRS  instru¬ 
ment  was  rather  inflexible  in  the  response  format  offered  to 
recruits.  All  questionnaire  items  (excepting  date,  social  secu¬ 
rity  number,  etc.)  were  precoded.  One  implication  of  optically 
scanning  such  a  questionnaire  is  that  any  deviation  (however 
valid)  from  preset  response  patterns  renders  the  response  as  out- 
of-range.  This  inflexibility,  however,  translates  into  a  rapid 
conversion  of  the  survey  into  a  machine-readable  form. 

By  contrast,  the  1985  NRS  instrument  was  quite  flexible  in 
response  format.  A  mixture  of  precoded  and  open-ended  questions 
were  asked.  Use  of  open-ended  questions  allowed  the  collection 
of  new  information  not  obtainable  in  OPSCAN  format.  In  addition, 
respondents  were  able  to  record  comments  and  clarifications  in 
the  margins  of  the  instrument.  As  this  instrument  was  manually 
coded  rather  than  optically  scanned,  these  comments  were  able  to 
be  used  in  resolving  questionable  responses  (e.g.,  MOS,  parents 
occupation,  education,  etc.).  The  tradeoff  here  was  that  the 
greater  amount  of  information  and  discretion  obtained  through  the 
use  of  a  traditional  survey  format  required  a  greater  expenditure 
of  time  and  effort  to  translate  responses  into  a  machine-readable 
form.  This  was  especially  true  when  open-ended  questions  had  to 
be  coded. 

How  completely  must  survey  responses  be  documented  and  how 
free  from  error  must  the  data  file  be?  The  answers  to  these 
questions  impact  greatly  on  the  total  survey  effort.  In  1985  the 
overriding  concerns  motivating  survey  procedures  were  those  of 
documentation  and  error  reduction.  These  requirements  necessi¬ 
tated  the  establishment  of  complex  and  extensive  survey  process¬ 
ing  procedures  (see  the  1985  User's  -Manual,  Research  Note  86-50). 
The  development  and  maintenance  of  survey  and  item  audit  trails 
was  costly  in  terms  of  both  time  and  personnel  effort.  It  did 
assure,  however,  that  the  survey  data  base  constructed  was 
extensively  documented  and  of  exceptionally  high  quality. 

The  same  cannot  be  said  of  the  1984  survey  data  base.  As 
this  administration  used  an  optical  scanning  answer  sheet,  data 
quality  is  solely  a  function  of  the  accuracy  of  responses  as 
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provided  by  respondents.  Because  no  coding  operation  was  imple¬ 
mented,  when  the  respondent  miscoded  an  answer  or  somehow  started 
answering  out  of  turn,  there  was  no  opportunity  for  recovering 
correct  responses.  Although  the  quality  of  the  1984  survey  data 
base  could  not  be  documented  in  the  same  manner  as  was  possible 
for  the  1985  NRS,  the  use  of  OPSCAN  considerably  reduced  the 
labor  required  for  the  construction  of  a  raw  (i.e.,  unedited) 
survey  database.  Because  audit  trails  were  not  established, 
surveys  passed  rapidly  from  collection  to  database  compilation. 

How  soon  following  survey  administration  must  data  be 
available  for  analysis?  As  we  have  seen  in  the  discussion  of  the 
two  issues  above,  considerable  difference  existed  in  the  opera¬ 
tionalization  of  the  1984  and  1985  NRS.  The  1984  NRS  used  a 
standard  precoded  OPSCAN  instrument  whereas  in  1985  a  more 
traditional  survey  instrument  was  used.  In  addition,  1984  data 
processing  procedures  included  no  manual  coding  while  extensive 
manual  processing  was  a  requirement  for  the  1985  administration. 
The  implications  for  turnaround  time  are  obvious.  The  1984 
OPSCAN  survey  required  only  one  step  (reading  of  the  answer 
sheets)  to  progress  from  responses  on  an  answer  sheet  to  a  raw 
data  file.  Many  more  steps  were  required  for  the  1985  NRS. 

What  is  the  level  of  effort  and  expenditure  available  for 
the  NRS?  Instrument  design  and  processing  constitute  only  two 
factors  in  the  determination  of  overall  survey  cost.  They  are 
important  factors,  however.  In  comparing  the  1984  and  1985 
administrations,  we  see  that  the  greatest  differences  between  the 
two  reside  in  their  respective  budget  categories.  Both  OPSCAN 
and  traditional  surveys  require  developmental  work  in  the  area  of 
instrument  development.  Regardless  of  the  format  adopted,  agree¬ 
ment  must  be  reached  as  to  the  questions  to  be  asked.  It  is 
following  this  agreement  that  OPSCAN  and  traditional  surveys 
diverge.  Traditional  surveys  implement  coding  and  keypunching 
operations  for  the  translation  of  survey  data  into  machine- 
readable  form.  As  discussed  above,  this  can  be  quite  a  consider¬ 
able  budgetary  item  if  open-ended  coding  and  audit  trails  are 
required.  OPSCAN  surveys  as  administered  for  the  1984  NRS  do  not 
require  such  data  handling  operations.  Instead,  they  require 
extensive  set-up  prior  to  the  reading  of  answer  sheets.  Before 
answer  sheets  may  be  read  by  optical  scanning  devices,  these 
devices  must  be  programmed  to  read  the  correct  answer  fields. 

This  is  not  an  inconsiderable  budgetr'item  as  initial  set-up  for  a 
single  answer  sheet  may  cost  thousands  of  dollars  before  the 
first  sheet  is  read. 

These  comments  have  served  to  characterize  the  1984  and  1985 
NRS  administrations  on  a  number  of  important  issues.  The  1984 
OPSCAN  administration  was  described  as  relatively  inflexible  with 
regard  to  the  data  collected  and  unable  to  be  fully  documented  in 
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the  area  of  data  quality.  In  contrast,  the  1985  survey  adminis¬ 
tration  utilized  a  more  flexible  survey  format  (including  open- 
ended  questions)  and  was  more  extensively  documented  and  pro¬ 
cessed.  On  the  other  hand,  the  potential  turnaround  time  from 
completed  survey  to  a  raw  data  file  was  much  faster  for  the 
OPSCAN  administration.  Only  machine-reading  was  required  in  this 
case  whereas  the  traditional  survey  implemented  extensive  coding, 
keypunching,  and  audit  processing.  In  the  area  of  cost  it  became 
apparent  that  the  major  differences  between  the  two  administra¬ 
tions  resided  in  the  budgetary  categories  required.  OPSCAN  as 
implemented  in  1984  required  no  open-ended  or  manual  consistency 
coding  and  so  realized  a  savings  in  manual  processing.  Tradi¬ 
tional  survey  processing  as  accomplished  in  1985  did  not  require 
the  extensive  programming  set-up  necessitated  by  OPSCAN.  In  the 
present  comparison,  the  cost  of  coding  operations  in  1985 
exceeded  the  programming  set-up  costs  incurred  in  the  1984  NRS. 

Although  these  characterizations  accurately  differentiate 
between  the  1984  and  1985  NRS  administrations,  they  should  not  be 
considered  as  the  only  options  available  for  either  OPSCAN  or 
traditional  surveys.  Each  may  be  modified  in  a  number  of  ways  to 
become  more  responsive  to  design  considerations  and  the  issues 
raised  above.  For  example,  it  was  noted  that  OPSCAN  surveys  are 
generally  relatively  inflexible  with  regard  to  question  response 
formats.  This  is  not  a  necessary  condition,  however.  It  is  pos¬ 
sible  to  include  open-ended  questions,  for  example,  in  an  OPSCAN 
questionnaire.  This  increase  in  flexibility,  though,  is  not 
gained  without  incurring  an  attendant  cost.  While  OPSCAN  may 
read  precoded  responses,  open-ended  responses  will  require  manual 
coding  much  as  required  for  traditional  survey  formats. 

Similar  tradeoffs  can  be  made  for  traditional  surveys. 

Rather  than  instituting  an  extensive  coding  and  audit  operation, 
surveys  may  be  only  scan-edited  for  glaring  errors.  Surveys 
judged  acceptable  can  then  be  quickly  routed  to  keypunching  and 
only  the  few  surveys  containing  the  most  obvious  of  errors 
retained  for  manual  coding.  Implementing  such  a  procedure  would 
have  the  effect  of  dramatically  cutting  database  turnaround  time 
and  placing  it  approximately  on  par  with  an  OPSCAN  operation. 

The  tradeoff  here  is  that  database  documentation  and  quality 
would  diminish  in  proportion  to  the  amount  of  processing 
eliminated. 

Obviously,  the  differences  between  OPSCAN  and  traditional 
formats  become  minimized  as  their  standard  implementations  are 
modified.  The  initial  strengths  of  OPSCAN  surveys  (quick 
turnaround  and  relatively  low  processing  costs)  become  less 
pronounced  as  additional  data  collection  features  (such  as  open- 
ended  questions)  are  added  to  the  design.  In  a  similar  manner, 
one  of  the  great  strengths  of  the  traditional  survey  design 
(control  and  documentation  of  the  data  preparation  operation)  is 
reduced  as  concessions  are  made  (e.g.,  scan-edits)  to  speed 
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turnaround  time.  Such  hybrid  designs  and  processing  operations 
as  those  noted  above  offer  a  number  of  options  for  future  NRS 
consideration. 


No  explicit  recommendations  are  made  here  regarding  the 
choice  between  OPSCAN  and  traditional  survey  formats.  What  we 
have  attempted  to  demonstrate,  rather,  are  issues  that  must  be 
addressed  during  the  survey  design  phase  of  the  NRS  and  their 
implications  for  the  instrument  and  its  processing.  The  final 
decision  to  adopt  one  or  another  particular  format  is  one  that 
must  be  administration-specific. 


NRS  Survey  Administration 

Recommendations  presented  in  this  section  concern  the 
administration  of  the  NRS.  Comparison  of  1984  and  1985  NRS  pro¬ 
cedures  has  led  to  the  formulation  of  the  following  two  general 
recommendations . 

•  NRS  administrative  schedules  and  personnel  responsibil¬ 
ities  should  be  clearly  established  and  maintained. 
Reception  Station  and  research  personnel  must  work 
together  to  implement  the  NRS  and  monitor  its  progress. 

•  Actual  administration  of  the  NRS  should  be  supervised 
bv  research  personnel — not  Reception  Station  personnel. 
On-site  supervision  by  research  personnel  is  needed  to 
maintain  the  cooperation  of  Reception  Station  personnel 
and  to  establish  quality  controls  for  survey 
administration  procedures. 

Conducting  a  survey  of  new  recruits  at  Army  Reception 
Stations  is  difficult.  Reception  Stations  are  highly  specialized 
facilities  designed  to  quickly  perform  the  initial  training, 
inspection,  testing,  and  processing  of  new  recruits  into  the 
Army.  Normally,  it  requires  three  days  to  complete  processing 
and  ship  a  recruit  cohort.  Each  day  begins  shortly  after  0400 
hours  and  proceeds  largely  without  interruption  until  lights  out 
shortly  after  2000  hours.  During  the  three  days  available  for 
processing.  Reception  Station  personnel  must  supervise  all 
recruit  activities  making  certain  that  schedules  and  appointments 
are  met.  Since  new  cohorts  arrive  each  working  day,  at  any  one 
time  Reception  Station  personnel  are  coordinating  the  processing 
of  three  cohorts  of  new  recruits.  Obviously,  conforming  to  the 
schedule  is  a  high  priority  at  Reception  Stations.  The  imposi¬ 
tion  of  any  break  in  timetables  (such  as  NRS  administration)  is 
bound  to  pose  some  difficulty.  The  two  recommendations  discussed 
below  are  offered  as  ways  to  lessen  the  disruption  an  NRS  admin¬ 
istration  can  effect  at  a  Reception  Station  while  insuring  the 
collection  of  complete  and  accurate  data. 
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Clearly  Establish  NRS  Scheduling  and  Personnel  Responsi¬ 
bilities.  It  is  vitally  important  to  the  success  of  any  NRS 
administration  that  schedules  and  personnel  responsibilities  be 
clearly  established  well  in  advance  of  actual  administration.  As 
a  first  step  in  this  process,  Reception  Station  commanders  should 
be  sent  an  introductory  letter  at  least  six  weeks  prior  to  the 
first  NRS  administration.  The  letter  should  include  a  brief 
summary  of  the  purpose  of  the  survey,  a  clear  statement  of  the 
survey's  utility,  and  a  request  for  the  commander  to  identify  a 
military  point  of  contact  (POC)  for  research  personnel. 

Following  identification  of  a  POC  for  the  NRS,  both  the 
Reception  Station  commander  and  the  POC  should  be  contacted  by 
research  personnel  in  order  to  discuss  requirements  (scheduling, 
space  for  survey  administration  and  questionnaire  storage, 
tracking  procedures  for  no-shows,  etc.),  and  the  respective 
responsibilities  of  Reception  Station  and  research  personnel. 
Since  each  Reception  Station  establishes  its  own  scheduling  for 
processing,  it  is  expected  that  arrangements  will  vary  across 
Reception  Stations.  Nonetheless,  it  is  important  that  they  be 
set  and  mutually  agreed  upon.  Most  likely  this  will  require  a 
series  of  conversations  and  letters  between  station  and  research 
personnel . 

Once  NRS  administrations  begin,  their  progress  must  be 
monitored.  Daily  accounting  of  the  number  of  recruits  surveyed 
as  well  as  no-shows  should  be  produced.  Without  this  information 
it  is  impossible  to  effectively  assess  the  completeness  of  NRS 
coverage.  The  identification  and  rescheduling  procedures  adopted 
for  no-shows  should  be  thoroughly  discussed  prior  to  survey 
administration.  That  is  one  area  where  the  pressure  on  Reception 
Station  personnel  to  process  and  ship  can  result  in  the  loss  of 
considerable  numbers  of  new  recruits. 

NRS  Should  be  Administered  bv  Research  Personnel.  There  is 
a  considerable  amount  of  work  associated  with  the  administration 
of  surveys  at  a  Reception  Station.  Before  actual  administration, 
physical  requirements  must  be  met  (i.e.,  a  room  for 
administration  and  space  for  questionnaire  storage) ,  recruits 
must  be  scheduled,  and  preparations  completed  (i.e.,  assembly  of 
materials  and  interleaving  of  survey  forms) .  During  the  survey, 
questionnaires  must  be  distributed,  instructions  given,  questions 
answered,  and  surveys  collected.  Following  administration,  sur¬ 
veys  must  be  inspected  for  completeness,  tallys  made  of  recruits 
surveyed,  no-shows  identified,  and  arrangements  made  for  the 
shipping  of  completed  questionnaires.  Finally,  the  appropriate 
personnel  must  be  contacted  for  the  rescheduling  of  no-shows. 

Experience  has  shown  that  it  is  unreasonable  to  expect 
Reception  Station  personnel  to  accomplish  these  tasks  in  addition 
to  their  normal  responsibilities.  Any  individual  assigned  to 
complete  all  these  tasks  would  not  be  available  for  normal  duty 
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and  the  use  of  multiple  personnel  raises  serious  coordination 
problems.  Fort  Dix  provides  a  good  example  of  the  gains  realized 
through  the  use  of  on-site  research  personnel  for  NRS  administra¬ 
tion  (see  the  Sample  Representativeness  discussion  in  the  1984 
User's  Manuals,  Research  Notes  86-46  and  86-47).  Table  2  reports 
the  percentage  of  total  NRS  surveys  contributed  by  Fort  Dix 
during  the  1984  and  1985  administrations. 


Table  2 


Fort  Dix 

Percentage  Contribution  to  Total  Number  of  Surveys 
Completed  by  NRS  Administration* 


Summer  1984 

Fall/Winter  1984 

Summer  1985 

% 

% 

% 

2.4 

5.1 

17.1 

*  Source:  Table  2,  1984  User's  Manual,  Research  Note  86-46? 
Table  1,  Supplementary  User's  Manual  for  October  84/ 

February  85  Administration,  Research  Note  86-47;  and  Table  3, 
1985  User's  Manual,  Research  Note  86-50. 


It  is  obvious  from  Table  2  that  in  1984  Fort  Dix  experienced 
difficulties  in  administering  the  NRS  using  only  Reception 
Station  personnel.  When  research  personnel  were  used  for  this 
purpose,  however,  NRS  implementation  was  considerably  more 
successful  (the  response  rate  for  Fort  Dix  in  1985  was  .936). 

It  has  been  our  experience  that  the  use  of  on-site  research 
personnel  enhances  the  efficiency  and  quality  of  NRS  adminis¬ 
tration  in  four  important  ways.  First,  interactions  between 
Reception  Station  and  research  personnel  foster  cooperation. 
Reception  Station  personnel  are  free  to  question  researchers 
about  the  NRS — its  purpose,  procedures,  and  implications  for 
them.  In  turn,  researchers  can  discuss  implementation  with  key 
Reception  Station  personnel  and  negotiate  a  schedule  that  will 
disrupt  normal  operations  minimally.  This  interaction  replaces 
compliance  anxiety  (What  do  I  have  to  do"  What  will  happen  if 
too  few  recruits  are  surveyed?)  with  an  atmosphere  where  respon¬ 
sibilities  and  procedures  are  known. 

Second,  the  presence  of  on-site  research  personnel  provides 
a  tangible  demonstration  of  the  importance  of  the  NRS.  Daily 
interaction  with  Reception  Station  personnel  (meetings  to  discuss 
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changes  in  survey  scheduling,  the  identification  of  no-shows, 
etc.)  assures  a  high  visibility  for  researchers.  This  presence 
underscores  the  seriousness  of  the  NRS .  Realizing  this  impor¬ 
tance,  Reception  Station  personnel  responded  in  a  professional 
manner.  Problems  (e.g.,  scheduling  mix-ups,  no-shows,  etc.) 
became  issues  to  resolve,  not  barriers. 

The  use  of  professional  researchers  also  standardizes  survey 
administration  procedures  in  a  way  not  possible  when  Reception 
Station  personnel  perform  this  function.  Research  personnel  are 
trained  to  use  a  "script"  which  assures  that  introductions, 
instructions,  and  responses  to  questions  will  be  uniform.  This 
ability  comes  from  a  thorough  understanding  of  the  purpose  of  the 
NRS,  detailed  familiarity  with  the  instrument,  and  a  knowledge  of 
conditions  which  may  affect  the  validity  of  responses.  Profes¬ 
sional  survey  administrators,  then,  enhance  data  quality  by 
providing  uniform  conditions  for  recruits. 

Finally,  the  use  of  on-site  research  personnel  facilitates 
the  documentation  of  survey  implementation.  Through  active  par¬ 
ticipation  in  the  survey  process,  researchers  are  in  a  position 
to  monitor  and  document  NRS  progress.  Reception  Station  person¬ 
nel  are  not  in  this  position.  Though  stations  monitor  their  own 
performance  daily,  they  are  not  designed  to  monitor  external 
(i.e.,  NRS)  production.  Having  daily  contact  with  a  variety  of 
Reception  Station  personnel,  researchers  are  able  to  assemble  the 
documentation  required  for  the  NRS.  The  importance  of  such  docu¬ 
mentation  is  demonstrated  by  the  fact  that  the  1985  administra¬ 
tion  was  the  first  reporting  survey  response  rate  figures. 


Summary 


New  Recruit  Surveys  (NRS)  provide  information  to  Army 
personnel  planners  regarding  the  characteristics,  knowledge  of 
enlistment  options,  and  enlistment  motivations  of  new  recruits. 
Methodologies  employed  in  recent  years  (1984  and  1985)  were 
examined  to  develop  an  understanding  of  the  strengths  and 
weaknesses  of  each.  In  this  way,  future  efforts  can  take 
advantage  of  the  experience  of  previous  NRS  administrations. 

Based  upon  the  lessons  learned,  four  recommendations  are 
made  for  the  conduct  of  future  surveys: 

•  The  survey  instrument  revision  process  should  be 
formalized; 


The  use  of  separate  answer  sheets  should  be 
discontinued; 


NRS  administration  schedules  and  personnel 
responsibilities  should  be  clearly  established  and 
maintained;  and 

Actual  administration  of  the  NRS  should  be  supervised 
by  research  personnel,  not  Reception  Station  personnel. 
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